A practical guide to EEG hyperscanning in joint action research: from motivation to implementation.

Developments in cognitive neuroscience have led to the emergence of hyperscanning, the simultaneous measurement of brain activity from multiple people. Hyperscanning is useful for investigating social cognition, including joint action, because of its ability to capture neural processes that occur within and between people as they coordinate actions toward a shared goal. Here, we provide a practical guide for researchers considering using hyperscanning to study joint action and seeking to avoid frequently raised concerns from hyperscanning skeptics. We focus specifically on Electroencephalography (EEG) hyperscanning, which is widely available and optimally suited for capturing fine-grained temporal dynamics of action coordination. Our guidelines cover questions that are likely to arise when planning a hyperscanning project, ranging from whether hyperscanning is appropriate for answering one's research questions to considerations for study design, dependent variable selection, data analysis and visualization. By following clear guidelines that facilitate careful consideration of the theoretical implications of research design choices and other methodological decisions, joint action researchers can mitigate interpretability issues and maximize the benefits of hyperscanning paradigms.

Understanding others' distal goals from proximal communicative actions.

Many social interactions require individuals to coordinate their actions and to inform each other about their goals. Often these goals concern an immediate (i.e., proximal) action, as when people give each other a brief handshake, but they sometimes also refer to a future (i.e. distal) action, as when football players perform a passing sequence. The present study investigates whether observers can derive information about such distal goals by relying on kinematic modulations of an actor's instrumental actions. In Experiment 1 participants were presented with animations of a box being moved at different velocities towards an apparent endpoint. The distal goal, however, was for the object to be moved past this endpoint, to one of two occluded target locations. Participants then selected the location which they considered the likely distal goal of the action. As predicted, participants were able to detect differences in movement velocity and, based on these differences, systematically mapped the movements to the two distal goal locations. Adding a distal goal led to more variation in the way participants mapped the observed movements onto different target locations. The results of Experiments 2 and 3 indicated that this cannot be explained by difficulties in perceptual discrimination. Rather, the increased variability likely reflects differences in interpreting the underlying connection between proximal communicative actions and distal goals. The present findings extend previous research on sensorimotor communication by demonstrating that communicative action modulations are not restricted to predicting proximal goals but can also be used to infer more distal goals.

Beyond Single-Mindedness: A Figure-Ground Reversal for the Cognitive Sciences.

A fundamental fact about human minds is that they are never truly alone: all minds are steeped in situated interaction. That social interaction matters is recognized by any experimentalist who seeks to exclude its influence by studying individuals in isolation. On this view, interaction complicates cognition. Here, we explore the more radical stance that interaction co-constitutes cognition: that we benefit from looking beyond single minds toward cognition as a process involving interacting minds. All around the cognitive sciences, there are approaches that put interaction center stage. Their diverse and pluralistic origins may obscure the fact that collectively, they harbor insights and methods that can respecify foundational assumptions and fuel novel interdisciplinary work. What might the cognitive sciences gain from stronger interactional foundations? This represents, we believe, one of the key questions for the future. Writing as a transdisciplinary collective assembled from across the classic cognitive science hexagon and beyond, we highlight the opportunity for a figure-ground reversal that puts interaction at the heart of cognition. The interactive stance is a way of seeing that deserves to be a key part of the conceptual toolkit of cognitive scientists.

Digital joint action: Avatar-mediated social interaction in digital spaces.

This article proposes a framework to characterize joint action in digital spaces. "Digital joint action" maintains many known elements from physical, real-world joint action including representations relating to joint goals and individual subgoals, processes such as predicting and monitoring own and others' actions, and supporting coordination through signaling and direct communication. In contrast to social interaction in the real world, joint action performed online comes with a unique additional feature: Digital joint action is mediated through (more or less vividly visualized) avatars that are controlled by the individual users but also imply particular personas that come with their own skills and acting abilities. This makes digital joint action a highly interesting research field as it allows to investigate the cognitive principles of joint action that lie outside of the constraints of human physicality but are nevertheless embodied (i.e., in a virtual body). The aim of this article is two-fold: First, we introduce digital joint action as joint action between avatars in a digital environment, and we specify commonalities and differences between joint actions in the real world and in digital spaces to provide a framework for further research. Second, using a survey study among users of the popular massively multiplayer online role-playing game (MMORPG) Final Fantasy XIV, we provide empirical validation for our approach.

Sharing Perspectives: Inviting Playful Curiosity Into Museum Spaces Through a Performative Score.

We report on the performative score "Sharing Perspectives" from the art/science research collaboration, Experimenting, Experiencing, Reflecting. Sharing Perspectives (SP) is developed as a score, inspired by choreography and the postmodern dance form Contact Improvisation, to stage exploration and improvisation, exploring uncertainty, creativity, togetherness, and the relationship between bodies and between bodies and space and artworks. The SP score acts as an experiment in how a brief intervention may affect the way art exhibitions are experienced, exploring how deeper and more sensorial engagement with art may be facilitated, for the benefit of visitors, galleries and artists. Based on questionnaires and qualitative interviews with participants during the Olafur Eliasson exhibition "In Real Life" at the Tate Modern in London in November 2019, we explore how the SP score modulates a playful mode of being, enhancing the experience of a museum art exhibition as a space of transformation and reflection. We find that the SP score encourages curiosity, which allows participants to recognize their habits for art and instead experience art slowly, recognize their comfort zones and move past them. As the score enacts a sensorial and playful approach to the exploration of the exhibition, participants experience a breaking of boundaries between each other, toward the other visitors, as well as to the artworks and the space itself, prompting an experience of being part of the exhibit as a whole. We discuss how the SP score invites a slowness and curiosity that takes on characteristics of play, which can change the participants' appreciation of an art space.

Crossmodal correspondences as common ground for joint action.

When performing joint actions, people rely on common ground - shared information that provides the required basis for mutual understanding. Common ground can be based on people's interaction history or on knowledge and expectations people share, e.g., because they belong to the same culture or social class. Here, we suggest that people rely on yet another form of common ground, one that originates in their similarities in multisensory processing. Specifically, we focus on 'crossmodal correspondences' - nonarbitrary associations that people make between stimulus features in different sensory modalities, e.g., between stimuli in the auditory and the visual modality such as high-pitched sounds and small objects. Going beyond previous research that focused on investigating crossmodal correspondences in individuals, we propose that people can use these correspondences for communicating and coordinating with others. Initial support for our proposal comes from a communication game played in a public space (an art gallery) by pairs of visitors. We observed that pairs created nonverbal communication systems by spontaneously relying on 'crossmodal common ground'. Based on these results, we conclude that crossmodal correspondences not only occur within individuals but that they can also be actively used in joint action to facilitate the coordination between individuals.

Motor constraints influence cultural evolution of rhythm.

While widely acknowledged in the cultural evolution literature, ecological factors-aspects of the physical environment that affect the way in which cultural productions evolve-have not been investigated experimentally. Here, we present an experimental investigation of this type of factor by using a transmission chain (iterated learning) experiment. We predicted that differences in the distance between identical tools (drums) and in the order in which they are to be used would cause the evolution of different rhythms. The evidence confirms our predictions and thus provides a proof of concept that ecological factors-here a motor constraint-can influence cultural productions and that their effects can be experimentally isolated and measured. One noteworthy finding is that ecological factors can on their own lead to more complex rhythms.

How does a partner's motor variability affect joint action?

Motor learning studies demonstrate that an individual's natural motor variability predicts the rate at which she learns a motor task. Individuals exhibiting higher variability learn motor tasks faster, presumably because variability fosters exploration of a wider space of motor parameters. However, it is unclear how individuals regulate variability while learning a motor task together with a partner who perturbs their movements. In the current study, we investigated whether and how variability affects performance and learning in such joint actions. Participants learned to jointly perform a sequence of movements with a confederate who was either highly variable or less variable in her movements. A haptic coupling between the actors led to translation of partner's movement variability into a force perturbation. We tested how the variability and predictability of force perturbations coming from a partner foster or hamper individual and joint performance. In experiment 1, the confederate produced more or less variable range of force perturbations that occurred in an unpredictable order. In experiment 2, the confederate produced more or less variable force perturbations in a predictable order. In experiment 3, the confederate produced more or less variable force perturbations in which the magnitude of force delivered was predictable whereas the direction of the force was unpredictable. We analysed individual performance, measured as movement accuracy and joint performance, measured as interpersonal asynchrony. Results indicated that in all three experiments, participants successfully regulated the variability of their own movements. However, individual performance was worse when partner produced highly variable force perturbations in an unpredictable order. Interestingly, predictability of force perturbations offset the detrimental effects of variability on individual performance. Furthermore, participants in the high variability condition achieved higher flexibility and resilience for a wide range of force perturbations, when the partner produced predictable movements. Participants improved their joint performance with a highly variable partner only when the partner produced partially predictable movements. Our results indicate that individuals involved in a joint action selectively rely on either their own or their partner's variability (or both) for benefitting individual and joint action performance, depending on the predictability of the partner' movements.

Informing, Coordinating, and Performing: A Perspective on Functions of Sensorimotor Communication.

Sensorimotor communication is a form of communication instantiated through body movements that are guided by both instrumental, goal-directed intentions and communicative, social intentions. Depending on the social interaction context, sensorimotor communication can serve different functions. This article aims to disentangle three of these functions: (a) an informing function of body movements, to highlight action intentions for an observer; (b) a coordinating function of body movements, to facilitate real-time action prediction in joint action; and (c) a performing function of body movements, to elicit emotional or aesthetic experiences in an audience. We provide examples of research addressing these different functions as well as some influencing factors, relating to individual differences, task characteristics, and situational demands. The article concludes by discussing the benefits of a closer dialog between separate lines of research on sensorimotor communication across different social contexts.

This is for you: Social modulations of proximal vs. distal space in collaborative interaction.

Human spatial representations are shaped by affordances for action offered by the environment. A prototypical example is the organization of space into peripersonal (within reach) and extrapersonal (outside reach) regions, mirrored by proximal (this/here) and distal (that/there) linguistic expressions. The peri-/extrapersonal distinction has been widely investigated in individual contexts, but little is known about how spatial representations are modulated by interaction with other people. Is near/far coding of space dynamically adapted to the position of a partner when space, objects, and action goals are shared? Over two preregistered experiments based on a novel interactive paradigm, we show that, in individual and social contexts involving no direct collaboration, linguistic coding of locations as proximal or distal depends on their distance from the speaker's hand. In contrast, in the context of collaborative interactions involving turn-taking and role reversal, proximal space is shifted towards the partner, and linguistic coding of near space ('this' / 'here') is remapped onto the partner's action space.

Combining Phase Advancement and Period Correction Explains Rushing during Joint Rhythmic Activities.

When people engage in rhythmic joint actions, from simple clapping games to elaborate joint music making, they tend to increase their tempo unconsciously. Despite the rich literature on rhythmic performance in humans, the mechanisms underlying joint rushing are still unknown. We propose that joint rushing arises from the concurrent activity of two separate mechanisms. The phase advance mechanism was first proposed in research on synchronously flashing fireflies and chorusing insects. When this mechanism is combined with a human-specific period correction mechanism, the shortened periods of individual intervals are translated into a tempo increase. In three experiments, we investigated whether joint rushing can be reliably observed in a joint synchronization-continuation drumming task. Furthermore, we asked whether perceptual similarities produced by the actions of different individuals modulate the joint rushing effect. The results showed that joint rushing is a robust phenomenon occurring in groups of different sizes. Joint rushing was more pronounced when the action effects produced by different individuals were perceptually similar, supporting the assumption that a phase advance mechanism contributed to rushing. Further control conditions ruled out the alternative hypothesis that rushing during rhythmic interactions can be explained by social facilitation or action mirroring effects.

Reciprocal information flow and role distribution support joint action coordination.

Many joint actions require task partners to temporally coordinate actions that follow different spatial patterns. This creates the need to find trade-offs between temporal coordination and spatial alignment. To study coordination under incongruent spatial and temporal demands, we devised a novel coordination task that required task partners to synchronize their actions while tracing different shapes that implied conflicting velocity profiles. In three experiments, we investigated whether coordination under incongruent demands is best achieved through mutually coupled predictions or through a clear role distribution with only one task partner adjusting to the other. Participants solved the task of trading off spatial and temporal coordination demands equally well when mutually perceiving each other's actions without any role distribution, and when acting in a leader-follower configuration where the leader was unable to see the follower's actions. Coordination was significantly worse when task partners who had been assigned roles could see each other's actions. These findings make three contributions to our understanding of coordination mechanisms in joint action. First, they show that mutual prediction facilitates coordination under incongruent demands, demonstrating the importance of coupled predictive models in a wide range of coordination contexts. Second, they show that mutual alignment of velocity profiles in the absence of a leader-follower dynamic is more wide-spread than previously thought. Finally, they show that role distribution can result in equally effective coordination as mutual prediction without role assignment, provided that the role distribution is not arbitrarily imposed but determined by (lack of) perceptual access to a partner's actions.

Co-actors represent the order of each other's actions.

Previous research has shown that people represent each other's tasks and actions when acting together. However, less is known about how co-actors represent each other's action sequences. Here, we asked whether co-actors represent the order of each other's actions within an action sequence, or whether they merely represent the intended end state of a joint action together with their own contribution. In the present study, two co-actors concurrently performed action sequences composed of two actions. We predicted that if co-actors represent the order of each other's actions, they should experience interference when the order of their actions differs. Supporting this prediction, the results of six experiments consistently showed that co-actors moved more slowly when performing the same actions in a different order compared to performing the same actions in the same order. In line with findings from bimanual movement tasks, our results indicate that interference can arise due to differences in movement parameters and due to differences in the perceptual characteristics of movement goals. The present findings extend previous research on co-representation, providing evidence that people represent not only the elements of another's task, but also their temporal structure.

When Height Carries Weight: Communicating Hidden Object Properties for Joint Action.

In the absence of pre-established communicative conventions, people create novel communication systems to successfully coordinate their actions toward a joint goal. In this study, we address two types of such novel communication systems: sensorimotor communication, where the kinematics of instrumental actions are systematically modulated, versus symbolic communication. We ask which of the two systems co-actors preferentially create when aiming to communicate about hidden object properties such as weight. The results of three experiments consistently show that actors who knew the weight of an object transmitted this weight information to their uninformed co-actors by systematically modulating their instrumental actions, grasping objects of particular weights at particular heights. This preference for sensorimotor communication was reduced in a fourth experiment where co-actors could communicate with weight-related symbols. Our findings demonstrate that the use of sensorimotor communication extends beyond the communication of spatial locations to non-spatial, hidden object properties.

Modulating action duration to establish nonconventional communication.

In many joint actions, knowledge about the precise task to be performed is distributed asymmetrically such that one person has information that another person lacks. In such situations, interpersonal coordination can be achieved if the knowledgeable person modulates basic parameters of her goal-directed actions in a way that provides relevant information to the co-actor with incomplete task knowledge. Whereas such sensorimotor communication has frequently been shown for spatial parameters like movement amplitude, little is known about how co-actors use temporal parameters of their actions to establish communication. The current study investigated whether systematic modulations of action duration provide a sufficient basis for communication. The results of 3 experiments demonstrate that knowledgeable actors spontaneously and systematically adjusted the duration of their actions to communicate task-relevant information if the naïve co-actor could not access this information in other ways. The clearer the communicative signal was the higher was the benefit for the co-actor's performance. Moreover, we provide evidence that knowledgeable actors have a preference to separate instrumental from communicative aspects of their action. Together, our findings suggest that generating and perceiving systematic deviations from the predicted duration of a goal-directed action can establish nonconventionalized forms of communication during joint action. (PsycINFO Database Record

Co-representation of others' task constraints in joint action.

Previous research has demonstrated that humans tend to represent each other's tasks even if no interpersonal coordination is required. The present study asked whether coactors in a joint action rely on task co-representation to achieve temporal coordination even if this implies increased movement effort for an unconstrained actor. Pairs of participants performed reaching movements back and forth between two targets, with the aim of synchronizing their landing times. One of the participants needed to move over an obstacle while the other had no obstacle. The results of four experiments showed that the participant without obstacle moved as if an obstacle was obstructing her way. Further amplifying the demands on interpersonal coordination led to a significant increase of this effect, indicating that unconstrained actors represented their coactor's task constraint and adjusted their own actions accordingly, particularly under high coordination demands. The findings also showed that unconstrained actors represented the object property constraining their coactor's movement rather than parameters of this movement. We conclude that joint action partners rely on task co-representation to achieve temporal coordination in a task with asymmetric task constraints. (PsycINFO Database Record

Let's decide together: Differences between individual and joint delay discounting.

This study addressed the question whether or not social collaboration has an effect on delay discounting, the tendency to prefer sooner but smaller over later but larger delivered rewards. We applied a novel paradigm in which participants executed choices between two gains in an individual and in a dyadic decision-making condition. We observed how participants reached mutual consent via joystick movement coordination and found lower discounting and a higher decisions' efficiency. In order to establish the underlying mechanism for dyadic variation, we further tested whether these differences emerge from social facilitation or inner group interchange.

How You Move Is What I See: Planning an Action Biases a Partner's Visual Search.

Activating action representations can modulate perceptual processing of action-relevant dimensions, indicative of a common-coding of perception and action. When two or more agents work together in joint action, individual agents often need to consider not only their own actions and their effects on the world, but also predict the actions of a co-acting partner. If in these situations the action of a partner is represented in a functionally equivalent way to the agent's own actions, one may also expect interaction effects between action and perception across jointly acting individuals. The present study investigated whether the action of a co-acting partner may modulate an agent's perception. The "performer" prepared a grasping or pointing movement toward a physical target while the "searcher" performed a visual search task. The performer's planned action impaired the searcher's perceptual performance when the search target dimension was relevant to the performer's movement execution. These results demonstrate an action-induced modulation of perceptual processes across participants and indicate that agents represent their partner's action by employing the same perceptual system they use to represent an own action. We suggest that task representations in joint action operate along multiple levels of a cross-brain predictive coding system, which provides agents with information about a partner's actions when they coordinate to reach a common goal.

Minimalist Social-Affective Value for Use in Joint Action: A Neural-Computational Hypothesis.

Joint Action is typically described as social interaction that requires coordination among two or more co-actors in order to achieve a common goal. In this article, we put forward a hypothesis for the existence of a neural-computational mechanism of affective valuation that may be critically exploited in Joint Action. Such a mechanism would serve to facilitate coordination between co-actors permitting a reduction of required information. Our hypothesized affective mechanism provides a value function based implementation of Associative Two-Process (ATP) theory that entails the classification of external stimuli according to outcome expectancies. This approach has been used to describe animal and human action that concerns differential outcome expectancies. Until now it has not been applied to social interaction. We describe our Affective ATP model as applied to social learning consistent with an "extended common currency" perspective in the social neuroscience literature. We contrast this to an alternative mechanism that provides an example implementation of the so-called social-specific value perspective. In brief, our Social-Affective ATP mechanism builds upon established formalisms for reinforcement learning (temporal difference learning models) nuanced to accommodate expectations (consistent with ATP theory) and extended to integrate non-social and social cues for use in Joint Action.

The role of shared visual information for joint action coordination.

Previous research has identified a number of coordination processes that enable people to perform joint actions. But what determines which coordination processes joint action partners rely on in a given situation? The present study tested whether varying the shared visual information available to co-actors can trigger a shift in coordination processes. Pairs of participants performed a movement task that required them to synchronously arrive at a target from separate starting locations. When participants in a pair received only auditory feedback about the time their partner reached the target they held their movement duration constant to facilitate coordination. When they received additional visual information about each other's movements they switched to a fundamentally different coordination process, exaggerating the curvature of their movements to communicate their arrival time. These findings indicate that the availability of shared perceptual information is a major factor in determining how individuals coordinate their actions to obtain joint outcomes.